Minimization of Transformed L_1 Penalty: Theory, Difference of Convex Function Algorithm, and Robust Application in Compressed Sensing

We study the minimization problem of a non-convex sparsity promoting penalty function, the transformed l1 (TL1), and its application in compressed sensing (CS). The TL1 penalty interpolates l0 and l1 norms through a nonnegative parameter a ∈ (0,+∞), similar to lp with p ∈ (0, 1]. TL1 is known in the statistics literature to enjoy three desired properties: unbiasedness, sparsity and Lipschitz continuity. We first consider the constrained minimization problem and prove the uniqueness of global minimizer and its equivalence to l0 norm minimization if the sensing matrix A satisfies a restricted isometry property (RIP) and if a > a∗, where a∗ depends only on A. Though result contains the well-known equivalence of l1 norm and l0 norm, in the limit a → +∞, the main difficulty is in treating the lack of scaling property of the TL1 penalty function. For a general sensing matrix A, we show that the support set of a local minimizer corresponds to linearly independent columns of A, and recall sufficient conditions for a critical point to be a local minimum. Next, we present difference of convex algorithms for TL1 (DCATL1) in computing TL1-regularized constrained and unconstrained problems in CS. The DCATL1 algorithm involves outer and inner loops of iterations, one time matrix inversion, repeated shrinkage operations and matrix-vector multiplications. For the unconstrained problem, we prove convergence of DCALT1 to a stationary point satisfying the first order optimality condition. Finally in numerical experiments, we identify the optimal value a = 1, and compare DCATL1 with other CS algorithms on two classes of sensing matrices: Gaussian random matrices and over-sampled discrete cosine transform matrices (ODCT). Among existing algorithms, the iterated reweighted least squares method based on L1/2 norm is the best in sparse recovery for Gaussian matrices, and the DCA algorithm based on L1 −L2 penalty is the best for ODCT matrices. We find that for both classes of sensing matrices, the performance of DCATL1 algorithm (initiated with L1 minimization) always ranks near the top (if not the top), and is the most robust choice insensitive to RIP (incoherence) of the underlying CS problems.